Evolutionary Prisoner's Dilemma game on the Newman- Watts networks 
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Maintenance of cooperation was studied for a two-strategy evolutionary Prisoner's Dilemma game where the 
players are located on a one-dimensional chain and their payoff comes from games with the nearest and next- 
nearest neighbor interactions. The applied host geometry makes possible to study the impacts of two conflicting 
topological features. The evolutionary rule involves some noise affecting the strategy adoptions between the 
interacting players. Using Monte Carlo simulations and the extended versions of dynamical mean-field theory 
we determined the phase diagram as a function of noise level and a payoff parameter. The peculiar feature of 
the diagram is changed significantly when the connectivity structure is extended by extra links as suggested by 
Newman and Watts. 
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Maintenance of cooperative behavior among selfish indi- 
viduals in biological and social systems is a progressively 
studied and challenging problem yj. The celebrated Pris- 
oner's Dilemma (PD) game is a widely applied mathemat- 
ical model illustrating the conflict between the individually 
rational (selfish) and globally useful (cooperative) behavior. 

In the original two-person (one-shot) game the players can 
follow one of the two strategies called cooperation (C) and 
defection (D). The player's payoff (or fitness) depends on 
their choice that is determined by the elements of a payoff 
matrix. To be more specific, for mutual cooperation each 
player receives the reward R, two defectors receive the pun- 
ishment P, whilst a cooperator and defector receive the suck- 
ers payoff S and the temptation T (to choose defection), re- 
spectively. For the PD game the payoffs satisfy the rank- 
ing: T > R > P > S. According to the assumption of 
the traditional game theory, players make rational decision 
to maximize their own income. Consequently, they should 
choose defection independently from the other player's de- 
cision. For the iterated PD game an additional constraint 
(namely 2R > T + P) is assumed to provide the highest total 
income for mutual cooperation. 

In the spatial evolutionary PD games N players are dis- 
tributed on a lattice with periodic boundary conditions. Fol- 
lowing one of the above mentioned pure strategies each player 
plays a game with her neighbors. These games are repeated 
meanwhile the players are allowed to modify their strategies 
in a way defined by the strategy update rule. For example, 
in the model suggested by Nowak and May |3] the players 
were located on the sites of a square lattice, they played a PD 
game with all neighbors for discrete times t — 1,2,..., and 
before the next time step each player has adopted the strategy 
of those player who received the highest accumulated pay- 
off in her neighborhood (including herself). In these cellular 
automaton type models the coexistence of the C and D strate- 
gies is provided by a frozen or oscillating pattern due to the 
dynamical balance between the opposite invasion processes 
affected by the payoffs. For synchronized strategy updates C 
invasion can occur along the horizontal or vertical interfaces 
separating the domains of the C and D strategies. On the con- 
trary, D invasions are favored along the irregular interfaces 
where defectors can exploit many neighboring cooperators. 



The average density of cooperators is reduced drastically for 

f — I 

asynchronous strategy updates |4] as well as for the introduc- 
tion of additional noise including irrational strategy adoptions 
1 5] (for a recent survey see Ref. |6]). 

In the last years the spatial models were generalized by 
locating players on the sites of a network (graph) whose 
edges connect each player with her co-players (neighbors) 
Investigations were also extended to differ- 
ent versions of small-world structures created from a one- or 
two-dimensional lattice by rewiring some portion of connec- 
tions ll^.Tsi lTii 15,IT6l] or by adding extra links to the spatial 
structure [81, [ij, [T^ [l9tl . The systematic comparison of the 



cited results is difficult because of the wide variety of dynam- 
ical rules used in the mentioned studies. Nevertheless, it is 
concluded generally that the introduction of inhomogeneities 
supports the maintenance of cooperation among selfish indi- 
viduals ifTsi I20I1 . An important progress has been made by 
Santos et at., who described a mechanism that provides rele- 
vant advantage for cooperators in the presence of connected 
hubs (players with a large number of neighbors) [1(11. 

Further exploring the possible effects of the interaction 
topology on the cooperation level we study a simple host 
structure where two conflicting topological features can be 
detected. It is well known that cooperators die out in one- 
dimensional structure (z — 2) for all stochastic dynamical 
rules favoring the adoption of the more successful strategies 
lfTol[l2l[T3ll . Similar result is predicted by the mean-field anal- 
ysis fd\ describing a system where the co-players are chosen 
randomly for both playing games and learning strategies. The 
former results explain why the Monte Carlo (MC) simulations 
have indicated the extinction of cooperators for a sufficiently 
larg e z dependent on the evolutionary rule(s) and payoffs 
lfToll2lll . In contrast, the systematic investigations have eluci- 
dated that the cooperation can be maintained for random se- 
quential updates because the one-site overlapping triangles (in 
regular connectivity structure at z = 4) support the spreading 
of cooperation in the low noise limit [22]. According to the 
MC simulations (performed on many different two- and three- 
dimensional lattices and other regular networks) this feature 
remains valid for the investigated spatial and non-spatial con- 
nectivity structures if the overlapping triangles span the whole 
graph. Following this avenue, a more rigorous analysis of this 
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feature requires to study those regular structures which can- 
not be spanned completely by the overlapping triangles or on 
which the overlapping triangles form a one-dimensional struc- 
ture favoring an evolution towards homogeneous states via a 
domain growing process. Now our attention is focused on 
the latter case by considering a simple connectivity structure 
combining the mentioned conflicting topological features that 
become relevant in the low noise limit. 

For this purpose we study a two-strategy evolutionary PD 
game on a one-dimensional lattice where the sites are con- 
nected to z = 4 neighbors. The systematic investigation of 
the effect of noise and payoffs on the density of coopera- 
tors in the stationary states requires accurate knowledge about 
what happens on a host graph where the overlapping trian- 
gles span the whole system but the one-dimensional feature is 
preserved. Due to this curiosity the resultant phase diagram 
differs significantly from those studied previously as detailed 
below. Furthermore we show that the mentioned unique be- 
havior is modified significantly when the small-world effect is 
switched on by adopting the construction suggested by New- 
man and Watts f^. 

In our model N players are located on the nodes a; of a 
network and they can follow the C or the D strategies. The 
spatial strategy distribution is described by a set of two-state 
site variables, i.e., Sa; = C or D. For later convenience the 
local states are denoted by two-dimensional unit vectors 



C 



and D 



(1) 



and in this notation the payoffs can be expressed with the 
aid of simple matrix algebra. The normalized income of the 
player at the site x obeys the following form: 



where s+ is the transpose of the state vector Sa;, and the sum- 
mation runs over ah the neighbors (^Ix) of player x i\^lx\ 
refers to the number of neighbors surrounding the site x). The 
pre-factor in Eq. |2] is applied to help the comparison of the 
resultant phase diagrams with other systems analyzed in pre- 
vious studies |l22i l24l Izstl. Due to its simplicity we use the 
rescaled payoff matrix suggested by Nowak et al. yj, i.e.; 



A = 



1< 6 < 2. 



(3) 



In this notation 1 < 6 < 2 denotes the strength of temptation 
to exploit the neighboring cooperator(s). 

The time evolution of the strategy distribution is modelled 
by the so-called pairwise comparison dynamics when a ran- 
domly chosen player x adopts the strategy of her randomly 
chosen neighbors (y) with a probability depending on the pay- 
off difference as 

^^[8, ^ Sy] = — ^ , (4) 

1 + exp [[Ua: - Uy)/K\ 

where K characterizes the strength of noise in the strategy 
adoption process allowing the players to make irrational deci- 
sions too 11261 113 mt]. In other words x player can adopt the 



strategy of y player even if Ux > Uy. Starting from a random 
initial state the above strategy adoption process is repeated un- 
til the system reaches a stationary states characterized by the 
average density p of cooperators dependent on the noise K, 
temptation b, and connectivity structure. Three different sta- 
tionary states can be distinguished when the number of sites 
tends to infinity. In the first case cooperators dominate the 
whole system (p = 1) if 6 < 6ci- On the contrary, for suffi- 
ciently high temptation (b > bc2) only the defectors can sur- 
vive. In general, there exists a coexistence region (0 < p < 1 
for bci < b < bc2) where both C and D survive in the sta- 
tionary state. In order to quantify both critical values of b our 
analysis is extended to the region of & < 1 belonging to the so- 
called Stag Hunt games representing another social dilemma 

k9]. 

Now our analysis is restricted to two types of connectivity 
structures illustrated in Fig. [l] The first structure is a linear 
chain for nearest and next-nearest interactions with periodic 
boundary conditions. The second structure is created from 
the first one by adding new links to the chain according to the 
Newman-Watts construction (t^] with some minor changes. It 
means that links are added without removing any of the origi- 
nal ones and each site has a maximum of five neighbors. The 
modified structure is characterized by the ratio p of the added 
and original links. The limitation of the maximum number 
of neighbors prevents the formation of large hubs and can 
help the (future) extension of the generalized mean-field tech- 
niques for inhomogeneous structures. 



(a) 



(b) 




FIG. 1 : Different connectivity structures for which the evolutionary 
PD game is studied, (a) One-dimensional chain with nearest and 
next-nearest neighbor interactions (z — 4). The second (equivalent) 
graph below illustrates the overlapping triangle structure, (b) One- 
dimensional chain with Newman-Watts small-world modification. 

The MC simulations are performed on systems containing 
N — lO'^ — 10^ sites. In many cases the simulations are started 
from a random initial state and after a suitable transient time 
we have determined the average density p of cooperators by 
averaging over a sampling time (with duration comparable to 
the transient time). The transient time varied from 2 x 10^ to 
10^ MC steps depending on the parameters (during 1 MC step 
each player has a chance once on average to modify her strat- 
egy). Besides this traditional method we used another (more 
efficient) technique to determine the critical point for the tran- 
sitions M ^ C and D ^ C. In the so-called growing seed 
method the simulations are started from a state where only 
one defector was present in the sea of cooperators and the sur- 
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vival probability of defectors is measured (by averaging over 
several thousand independent runs) at different b values for a 
fixed K (for details see Ref. Q). 

Beside the MC simulations, the above model is investigated 
also by the application of the generalized mean-field (GMF) 
method. Within this approach the translation invariant sys- 
tem is characterized by a set of configuration probabilities on 
a compact cluster of n neighboring sites. This approximation 
involves the derivation of the hierarchy of equations of motion 
for the configuration probabilities on clusters of n sites (for 
details see Appendix in Ref. |6]). Clearly, the accuracy of this 
method can be improved by increasing the values of n. In the 
present model we have found that the qualitative prediction of 
GMF for the phase diagram does not change if n > 5. The 
most accurate approximation is achieved for n = 10. The cor- 
responding predictions are compared with MC data in Fig. 121 
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FIG. 2: Phase boundaries on the b — K plane for the one-dimensional 
connectivity structure with z = 4. Solid lines indicate the prediction 
of the generalized mean-field technique for n = 10 while connected 
symbols stand for MC results. The phase boundaries separate the 
homogeneous phases of cooperators (C) or defectors (D), and the 
mixed state M where cooperators and defectors coexist. The tricriti- 
cal points are indicated by arrows and also by full circles for the MC 
results and the GMF method. 

Figure |2] illustrates that both techniques show a step like 
transition from C to D phase at 6 = 1, that is bd — 6c2 = 1, 
in the limit A' ^ 0. A similar behavior can be concluded if 
K oo, that is resembling the mean-field prediction men- 
tioned above. First order transition from C to D is found 
(i.e., bci = bc2 > 1) when varying the value of temptation 
b for fixed noise level if its value exceeds a threshold value 
(K > k[^1^^ « 0.15). For lower noise level the first or- 
der transition splits into two continuous critical transitions be- 
longing to the directed percolation universality class liBOil as 
detailed below. For this threshold value of noise {K = Ktric) 
one can observe a tricritical point {b — b\^[^^ k, 1.05) where 
all the mentioned three phases can coexist. 

The continuous extinction process of both the C and D 
strategies exhibits universal behavior belonging to the directed 
percolation universality class [31]. This means that the vari- 



ation of b or K (as a control parameter) yields a power law 
decrease of order parameter (here the density of C or D strate- 
gies) when approaching the critical point. The algebraic be- 
havior is accompanied by a power law divergency in the corre- 
lation length, relaxation time, and fluctuation of order param- 
eter that makes the numerical analysis time consuming Isoll . 
In the critical point the order parameter decays algebraically 
as 

p{t) oc r'^ (5) 

for sufficiently long times. The value of a and other critical 
exponents are universal (independent of the details of dynam- 
ical rules) while their values are related to each other and de- 
pend on the spatial dimension. For a one-dimensional stochas- 
tic cellular automaton a = 0.159464(6) is found |32]. In or- 
der to demonstrate the same universal behavior, the quantity 
pt" is plotted as a function of time in Fig. [3] where 30 inde- 
pendent runs of iV = 2 X 10^ system size are averaged. 
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FIG. 3: Density decay of cooperators at 6 = 1.03 for three different 
noise levels (K = 0.0791, 0.0790, and 0.0789 from top to bottom). 
The middle curve illustrates the typical behavior in the critical point 
(Q = 0.159). 

Notice that in Fig. |2] the MC results are reproduced quali- 
tatively well by the GMF approximation. Quantitatively good 
agreement is found in the function bc2{K) for arbitrary value 
of K while for the lower phase boundary the reproduction is 
not satisfactory (if K < Ktric)- The relevance of long-range 
correlations in the extinction process of D strategies can ex- 
plain the difference in the prediction of GMF and MC results. 

The b—K phase diagram demonstrates another striking fea- 
ture of the evolutionary PD game for this connectivity struc- 
ture. Namely, Fig. |2] refers to the existence of an optimum 
noise level providing the best condition for the cooperators to 
survive. At the same time, when increasing K for b = 1.03 
one can observe three consecutive phase transitions, namely, 
D ^ M ^ C — *■ D. The smoothed version of this behavior 
was reported for those connectivity structures that cannot be 
spanned by overlapping triangles supporting the spreading of 
cooperation in the low noise limit for the pairwise comparison 
dynamical rule [,2Z1 . 
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FIG. 4: The lower 6ci(^^) and upper bc2{K) critical points on the 
Newman-Watts structure for different p values. The symbols corre- 
spond to p = 0.02 (open circle), 0.06 (full circle), and 0.10 (trian- 
gle). Solid lines are guide to the eye and dashed line indicates a base 
line at fo = 1. 



Interestingly, one can observe another re-entrance transi- 
tion along the boundary separating the C and M phases in 
Fig. |2] that is similar to previously observed coherence reso- 
nance phenomenon for the PD game [21] or even for other 
games [33]. 

As mentioned above the cooperation can be maintained 
even at the low noise limit (if b > 1) for many other connec- 
tivity structures spanned by overlapping triangles. The present 
study has demonstrated that this general trend is defeated by 
the additional one-dimensional constraint providing that prop- 
agating fronts cannot avoid each other on this structure. How- 
ever, the latter feature is destroyed when new links are added 
to the connectivity structure as demonstrated in Fig.[T| 



The effect of new links on the b ~ K phase diagram is il- 
lustrated in Fig. |4] for different values of p characterizing the 
portion of added links. From the present numerical results one 
can conclude that the region of the M state becomes wider 
as p increases and persists in the whole range of K we stud- 
ied. Simultaneously the tricritical point disappears immedi- 
ately and the lower phase boundary bci{K) remains less than 
1 if ^ as we introduce the additional links. The grad- 
ual variations are related to the emergence of the small-world 
character and simultaneously to the destruction of the strict 
one-dimensional properties when p is increased. On the con- 
trary, the upper phase boundary (between the D and M phases) 
tends to 6 = 1 as A' goes to if p does not exceed a thresh- 
old value ptr = 0.04(2). Unfortunately, the more accurate 
determination of ptr is prevented by the long relaxation time 
diverging if K ^ Q. 

To sum up, we have systematically studied the effect of 
noise K and temptation b (to choose defection) on the measure 
of cooperation for an evolutionary Prisoner's Dilemma game 
on a one-dimensional chain with nearest and next-nearest 
neighbor interactions. The application of this connectivity 
structure is motivated by the claim to clarify the possible im- 
pacts of elementary topological properties. Here the conflict- 
ing topological features result in a curious b — K phase di- 
agram where three phases can exist at special values of pa- 
rameters. It is also demonstrated that the basic features of the 
phase diagram are modified drastically when new links are 
added to the connectivity structure according to the Newman- 
Watts construction. 
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